Measuring Research Data Uncertainty in the 2010 NRC Assessment of Geography Graduate Education

This article characterizes and measures errors in the 2010 National Research Council (NRC) assessment of researchdoctorate programs in geography. This article provides a conceptual model for data-based sources of uncertainty and reports on a quantitative assessment of NRC research data uncertainty for a particular geography doctoral program. Findings indicate that important variables, including faculty totals and allocations and publication counts, are substantially undercounted, with important and negative impacts on program research activity measures. Further, these research measures are highly sensitive to small changes in counts and are particularly problematic for interdisciplinary fields such as geography. We caution against using the 2010 NRC data or metrics for any assessment-oriented study of research productivity.